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DETAILED ACTION 

Information Disclosure Statement 

1 . The Examiner has considered the references listed in the Information Disclosure 
Statement dated 9/02/03. A copy of the Information Disclosure Statement is attached 
to this office action. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

2. Claims 1, 7, 12 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Montgomery (UK Patent Application GB 2183880 A), hereinafter referred to as 
Montgomery. 

Regarding claim 1, Montgomery discloses a speech translator for the deaf. 
Montgomery's invention includes the following: 

• a modem that connects with the network to convey information to, and receive 
information from the network (Fig. 6, and Fig. 7, data/voice line int; Fig. 8); 

• a subscriber terminal having an interface that enables communication with the 
modem (Figs. 6 and 7. p. 2, lines 93-1 14, Speech translator for the deaf (STD) 
connected to modem), 
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• a display interface that communicates with a visual display device to display 
information (p. 1 , lines 25-30, STD text is displayed; Fig. 3, item 7), 

• a telephone interface that enables communication with a telephone to convey voice 
information of a user (Figs, 6, 7 and 8), and 

. a buffer that receives and stores speech information (Fig. 3, item 5; p. 1 , lines 105- 
111; recognition requires 40 to 1 50 small parts of speech, inherently requiring 
buffering); and 

• a processor to decode and display on the display device speech information as text 
upon receipt of speech information from the network (p. 1, lines 105-124; Fig. 3, items 
5, 6 and 7; recognizer produces symbolic text which is displayed). 

Regarding claim 7, Montgomery discloses a speech translator for the deaf. 
Montgomery's invention includes the following steps: 

• Receiving at a broadband telephony interface speech packets destined for the 
hearing impaired individual (Figs. 6, 7 and 8, data/voice line int; the connection to the 
deaf user's home can be over a voice/data lines such as LADT or ISDN [which supports 
speech packets] with the appropriate interface; p. 2, lines 100-104); 

. storing the speech packets in a buffer (Fig. 3, item 5; p. 1 , lines 105-1 1 1 ; recognition 
requires 40 to 150 small parts of speech, inherently requiring buffering); and 

• processing the speech packets to display textual representations thereof on a 
display device (p. 1, lines 105-124; Fig. 3, items 5, 6 and 7; recognizer produces 
symbolic text which is displayed). 
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Regarding claim 12, Montgomery discloses a speech translator for the deaf. 
Montgomery's invention includes the following: 

• a network interface that enables communication with the network (p. 2, lines 100- 
115; Fig. 8, data/voice line int); 

• a subscriber terminal that communicates information with the network interface a 
display device, and a telephone device or other auditory device (Fig, 8, shows terminal 
with display and telephone); and 

• a processor that decodes and displays speech information as text on the display 
device during receipt of real time speech information from the network (p. 1 , lines 105- 
124; Fig. 3, items 5, 6 and 7; recognizer produces symbolic text which is displayed) and 

• encodes and/or transmits speech information to the network when speech 
information is received from the telephone (Fig. 8; Fig. 3, item 1, microphone, 

TELEPHONE INTERFACE). 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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3. Claims 2, 8 and 13, are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Montgomery in view of Comerford et al. (U.S. Patent 6,107,935), hereinafter 
referred to as Comerford. 

Regarding claim 2, Montgomery teaches everything claimed, as applied above 
(see claim 1). But Montgomery does not specifically teach the following: "a memory that 
stores voice patterns, and wherein said processor further includes a speech analyzer 
that recognizes an incoming voice pattern based on information stored in the memory." 
However, the examiner contends that this concept was well known in the art, as taught 
by Comerford. 

In the same field of endeavor, Comerford discloses a speaker and speech 
recognition system. Comerford's system includes a speaker recognition module with 
stored voice patterns where a speaker's utterance is compared with existing data (Fig. 1 
and 2; col. 3, lines 55-65; col. 4, lines 15-22). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Comerford, because it is well known in the art at the time of 
invention for the purpose of accurately identifying the speaker of an utterance 
(Comerford, col. 1 , lines 10-60), which is of particular value to a deaf user. 

Regarding claim 8, Montgomery teaches everything claimed, as applied above 
(see claim 7). But Montgomery does not specifically teach the following: 
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a) storing speech patterns in a database, and 

b) analyzing and comparing incoming speech obtained by processing the speech 
packets with speech patterns stored in the database in order to provide speaker 
identification capability. However, the examiner contends that these concepts were well 
known in the art, as taught by Comerford. 

In the same field of endeavor, Comerford discloses a speaker and speech 
recognition system. Comerford's system includes a speaker recognition module with 
stored voice patterns where a speaker's utterance is compared with existing data, a) 
above (Fig. 1 and 2, stored speech patterns for speakers, created during training, col. 4, 
lines 15-25, a), above; col. 3, lines 55-65, speaker identification performed, b), above). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Comerford, because it is well known in the art at the time of 
invention for the purpose of accurately identifying the speaker of an utterance 
(Comerford, col. 1 , lines 1 0-60), which is of particular value to a deaf user. 

Regarding claim 13, Montgomery discloses a speech translator for the deaf. 
Montgomery's invention includes the following: 

• a subscriber terminal having a network interface that enables communication with a 
network (Figs. 6 and 7. p. 2, lines 93-1 14, Speech translator for the deaf (STD) 
connected to modem), 
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• a display interface that communicates with a visual display device to display textual 
information (p. 1, lines 25-30, STD text is displayed; Fig. 3, item 7), and 

• a telephone interface that enables communication with a telephone to convey voice 
information of a user (Figs, 6, 7 and 8),, 

• said subscriber terminal including a processor utilizing a speech buffer to receive at 
least one of streamed and real time speech information and to decode and display 
speech information as text on the display device during receipt of speech information 
from the network (Fig. 3, items 5, 6 and 7; p. 1 , lines 105-124; small parts of speech 
with inherent buffering converted to symbolic text which is displayed). 

But Montgomery does not specifically teach "a database that enables 
identification of a prior caller based on speech segments stored in a database." 
However, the examiner contends that this concept was well known in the art, as taught 
by Comerford. 

In the same field of endeavor, Comerford discloses a speaker and speech 
recognition system. Comerford's system includes a speaker recognition module with a 
database of stored voice patterns where a speaker's utterance is compared with 
existing data (Fig. 1 and 2; col. 3, lines 55-65; col. 4, lines 15-22). 

Therefore, it would have been obvious to one having ordinary skill in the art. at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Comerford, because it is well known in the art at the time of 
invention for the purpose of accurately identifying the speaker of an utterance 
(Comerford, col. 1, lines 10-60), which is of particular value to a deaf user. 
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4. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Montgomery in view of Comerford and further in view of Saindon et al. (U.S. Patent 
Application Publication 2002/0161579 A1), hereinafter referred to as Saindon. 

Regarding claim 4, Comerford teaches everything claimed, as applied above 
(see claim 1 ). But Comerford does not specifically teach "said subscriber terminal 
includes a speech database for storing speech segments identified with certain users, 
and said processor accesses said database to identify .... users according to matches 
between speech segments received in real time and stored in the database. However, 
the examiner contends that this concept was well known in the art, as taught by 
Comerford. 

In the same field of endeavor, Comerford discloses a speaker and speech 
recognition system. Comerford's system includes a speaker recognition module with 
stored voice patterns where a speaker's utterance is compared with existing data (Fig. 1 
and 2; col. 3, lines 55-65; col. 4, lines 15-22). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Comerford, because it is well known in the art at the time of 
invention for the purpose of accurately identifying the speaker of an utterance 
(Comerford, col. 1 , lines 10-60), which is of particular value to a deaf user. 
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Furthermore, Comerford does not specifically teach " said processor accesses 
said database to ... display the identity of users." However, the examiner contends 
that this concept was well known in the art, as taught by Saindon. 

In the same field of endeavor, Saindon discloses a system for automated audio 
transcription, translation, and transfer, where the names of the speakers are displayed 
with the text transcription of their spoken audio fl|'s[0130] and [0012]; Figs. 2 and 4). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Saindon, because it is well known in the art at the time of 
invention for the purpose of keeping track of who spoke a particular utterance of 
particular value to a deaf user. 

5. Claims 3, 9, and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Montgomery in view of Dietz (U.S. Patent 6,175,820), hereinafter referred to as 
Dietz. 

Regarding claim 3, Montgomery teaches everything claimed, as applied above 
(see claim 1), but Montgomery does not specifically teach "said processor includes a 
tonal and inflection analyzer that analyzes segments of speech in the buffer to modify 
visual characteristics of decoded speech information displayed on the display interface." 
However, the examiner contends that this concept was well known in the art, as taught 
by Dietz. 
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In the same field of endeavor, Dietz discloses a device for capturing voice 
dynamics to enhance speech-to-text. Dietz's system analyzes inflection and tone and is 
capable of determining some punctuation (col. 5, lines 54-60). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Dietz, because it is well known in the art at the time of invention 
for the purpose of enhancing the recognition of speech with marked up text (Dietz, col. 
2, lines 39-47). 

Regarding claim 9, Montgomery teaches everything claimed, as applied above 
(see claim 7), but Montgomery does not specifically teach "analyzing characteristics of 
incoming speech obtained by processing the speech packets and inserting punctuation 
in displayed textual representations thereof in response to the analysis." However, the 
examiner contends that this concept was well known in the art, as taught by Dietz. 

In the same field of endeavor, Dietz discloses a device for capturing voice 
dynamics to enhance speech-to-text. Dietz's system captures speech (Fig. 4, item 
402), analyzes inflection, tone, and other speech dynamics (col. 5, lines 57-60) and is 
capable of determining some punctuation (col. 5, lines 54-60). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Dietz, because it is well known in the art at the time of invention 
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for the purpose of enhancing the value of speech-to-text conversion by including text 
marked with punctuation (Dietz, col. 2, lines 39-47). 

Regarding claim 10, Montgomery in view of Dietz teaches everything claimed, as 
applied above (see claim 9); in addition, as stated previously (see rejection of claim 9) 
Dietz teaches "said characteristics include at least one of changes in tone, volume, and 
inflection" (col. 5, lines 57-60). 

6. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Montgomery in view of Comerford and further in view of Dietz. 

Regarding claim 15, Montgomery discloses a speech translator for the deaf. 
Montgomery's invention includes the following steps: 

• receiving real time speech information (p. 1, lines 27-29; speech sounds coming in 
over the telephone connection); 

• converting the real time speech information into text (p. 1 , lines 27-32; speech to 
symbolic text). 

But Montgomery does not specifically disclose "analyzing the speech information 
to determine identity of a caller based on previously stored speech segments." 
However, the examiner contends that this concept was well known in the art, as taught 
by Comerford. 
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In the same field of endeavor, Comerford discloses a speaker and speech 
recognition system. Comerford's system includes a speaker recognition module with 
stored voice patterns where a speaker's utterance is compared with existing data (Fig. 1 
and 2; col. 3, lines 55-65; col. 4, lines 15-22). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Comerford, because it is well known in the art at the time of 
invention for the purpose of accurately identifying the speaker of an utterance 
(Comerford, col. 1 , lines 10-60) of particular value for a deaf user. 

Furthermore, Montgomery does not teach "displaying at least one of textual 
representation of the speech, punctuation, obtained as a result of the analyzing step." 
However, the examiner contends that this concept was well known in the art, as taught 
by Dietz. 

In the same field of endeavor, Dietz discloses a device for capturing voice 
dynamics to enhance speech-to-text. Dietz's system captures speech (Fig. 4, item 
402), analyzes inflection, tone, and other speech dynamics (col. 5, lines 57-60) and is 
capable of determining some punctuation (col. 5, lines 54-60). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Dietz, because it is well known in the art at the time of invention 
for the purpose of enhancing the value of speech-to-text conversion by including text 
marked with punctuation (Dietz, col. 2, lines 39-47). 
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7. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Montgomery in view of Comerford and further in view of Iwamida (U.S. Patent 
5,839,109), hereinafter referred to as Iwamida. 

Regarding claim 14, this claim has limitations similar to claim 13 (rejected with 
Montgomery in view of Comerford), and these limitations are rejected for the same 
reasons. But Montgomery does not specifically teach "said processor further including 
a tonal and inflection analyzer that effects analysis of speech based on characteristics 
including at least one of gender, soft-spoken words, hard-spoken words, shouting, 
laughter, or human expression." However, the examiner contends that this concept 
was well known in the art, as taught by Iwamida. 

In the same field of endeavor, Iwamida discloses a speech recognition apparatus 
capable of recognizing signals other than spoken words and displaying them for viewing 
(title). These sounds include a human expression (col. 4, lines 43-47, a baby crying) 
and are analyzed by a time series of frequency feature parameters (col. 3, lines 50-57, 
which necessarily includes tonal variations). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Iwamida, because it is well known in the art at the time of 
invention for the purpose of presenting a broader range of sound information, which 
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would be of particular value to a hearing impaired person (Iwamida, col. 2, lines 13-17; 
col. 6, lines 32-36). 

8. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Montgomery in view of Comerford, Iwamida, and further in view of Dietz. 

Regarding claim 16, this claim has limitations that are similar to claim 15 
(rejected with Montgomery in view of Comerford and Dietz), and the corresponding 
limitations are rejected for the same reasons. But Montgomery does not specifically 
teach "analyzing the speech information to determine ... at least one of gender, soft- 
spoken words, hard-spoken words, shouting, laughter, or human expression." 
However, the examiner contends that this concept was well known in the art, as taught 
by Iwamida. 

In the same field of endeavor, Iwamida discloses a speech recognition apparatus 
capable of recognizing signals other than spoken words and displaying them for viewing 
(title). These sounds include a human expression (col. 4, lines 43-47, a baby crying) 
and are analyzed by a time series of frequency feature parameters (col. 3, lines 50-57, 
which necessarily includes tonal variations). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Iwamida, because it is well known in the art at the time of 
invention for the purpose of presenting a broader range of sound information, which 
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would be of particular value to a hearing impaired person (Iwamida, col. 2, lines 13-17; 
col. 6, lines 32-36). 

9. Claims 5, 6, and 1 1 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Montgomery in view Engelke et al. (U.S. Patent 6,075,842), hereinafter referred to 
as Engelke. 

Regarding claim 5, Montgomery teaches everything claimed, as applied above 
(see claim 1). In addition, Montgomer teaches the use of a keypad for entering 
commands to control functions within the system, including commands in the DTMF 
format (p. 2, lines 8-15; Fig. 3, items 4, 6, 9, 10, 1 1 , and 12), but Mongomery does not 
specifically teach "said processor includes a detector that responds to subscriber inputs 
to activate and deactivate speech recognition." However, the examiner contends that 
this concept was well known in the art, as taught by Engelke. 

In the same field of endeavor, Engelke discloses a method for text enhanced 
telephony. Engelke's method supports the extraction and display of coded text signals 
and includes a bypass circuit to disable the text processing circuitry (Fig. 1 , item 
40; col. 4, lines 57-61). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by expanding the DTMF control 
functions to include the ability to enable and disable the speech processing, as taught 
by Engelke, because it is well known in the art at the time of invention for the purpose of 
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saving power when the additional processing is not required as when a normal hearing 
person is using the system. 

Regarding claim 6, Montgomery in view of Engelke teaches everything claimed, 
as applied above (see claim 5); in addition, Montgomery teaches "said detector 
comprises a DTMF tone detector and said user inputs comprise DTMF tones of a 
telephone" (see claim 5 rejection). 

Regarding claim 11, Montgomery teaches everything claimed, as applied above 
(see claim 7), but Montgomery does not specifically teach "responding to a command 
from the subscriber to activate and deactivate speech processing." However, the 
examiner contends that this concept was well known in the art, as taught by Engelke. 

In the same field of endeavor, Engelke discloses a method for text enhanced 
telephony. Engelke's method includes the extraction and display of coded text signals 
and includes a bypass circuit to disable the text processing circuitry (Fig. 1, item 40; col. 
4, lines 57-61). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Montgomery by specifically providing the 
features, as taught by Engelke, because it is well known in the art at the time of 
invention for the purpose of saving power when the additional processing is not 
required; for example, when a normal hearing person was using the system. 



Application/Control Number: 10/053,777 Page 17 

Art Unit: 2654 

Citation of Pertinent Art 

10. The following prior art made of record but not relied upon is considered pertinent 
to the applicant's disclosure: 

• M.Bazzani, et al., ("PC-Based Telephone Communication System for Deaf-Blind 
People", Proc. of IEEE GLOBECOMM, Global Telecommunication Conference, 
Hollywood, Florida, Nov.1988) teach the use of a telephone interface with speech 
recognition as an aid to deaf-blind people. 

. Engelke et al. (U.S. Patent Application Publication 2003/01 25952 A1 ) discloses a 
voice and text transmission system using encoding and decoding over a network. 

• Ortega et al. (U.S. Patent 6,332,122) disclose a transcription system for multiple 
speakers using established identification. 

• Sharman et al. (U.S. Patent 6,100,882) disclose a system for textual recording of 
contributions to an audio conference using speech recognition. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to V. Paul Harper whose telephone number is 703 305- 
4197. The examiner can normally be reached on M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 703 305-9645. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



1/26/2005 



V. Paul Harper 
Examiner 
Art Unit 2654 




